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DETAILED ACTION 



1 . This office action is in response to application 10/812,000 filed March 30, 2004. 
Claims 1-20 are pending in the application and have been examined. 

Priority 

2. This application claims priority to Japanese application 2003-1 54625 filed on May 
30, 2003. This priority date has been considered in this office action. 

Information Disclosure Statement 

3. The Information Disclosure Statement filed March 30, 2004 has been accepted 
and considered in this office action. 



Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 



Claim 20 is rejected under 35 U.S.C. 101 because the claimed invention is directed to 
non-statutory subject matter. Claim 20 attempts to claim a machine-readable medium. 
However a machine-readable medium can be interpreted as merely a magnetic carrier 
wave, something that is considered to be non-statutory subject matter. Therefore claim 
20 is rejected as being non-statutory under 35 U.S.C. 101. 



Claim Rejections - 35 USC § 101 



4. 



35 U.S.C. 101 reads as follows: 
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Claim Rejections - 35 USC § 102 

5. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
states. 

6. Claim 1, 13, and 20 are rejected under 35 U.S.C. 102(b) as being anticipated by 
Nagata (A Stochastic Japanese Morphological Analyzer Using a Forward-DP 
Backward-A N-Best Search Algorithm). 

7. Consider claim 1 , Nagata teaches a morphological analyzer comprising 
(Japanese morphological analyzer, abstract): 

a hypothesis generator for applying a prescribed method of morphological 
analysis to a text and generating one or more hypotheses as candidate results of the 
morphological analysis, each hypothesis being a word string with part-of-speech tags, 
the part-of-speech tags including form information for parts of speech having forms (We 
propose a novel search strategy for getting the N best morphological analysis 
hypotheses for the input sentence; page 201 , column 1 , line 35. We used tile simple tri- 
POS model as the tagging model for Japanese; page 201 , column 1 , line 30.); 

a model storage facility storing information for a plurality of part-of-speech n- 
gram models, at least one of the part-of-speech n-gram models including information 
about the forms of the parts of speech (We used tile simple tri-POS model as the 
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tagging model for Japanese; page 201, column 1, line 30. Tagging model also 
described in detail in section 2. It is inherent that the model must be stored in a memory 
in order to enable the analyzer.); 

a probability calculator for finding a probability that each said hypothesis will 
appear in a large corpus of text by using a weighted combination of the information for 
the part-of-speech n-gram models stored in the model storage facility (equations 3, 4 
and 5 described in section 201 estimates the probabilities of the tagging with relation to 
the relative frequencies of the corresponding events.); and 

a solution finder for finding a solution among said hypotheses, based on the 
probabilities generated by the probability calculator (Once word hypotheses for 
unknown words are generated, the proposed N-best algorithm will find tile most likely 
word segmentation and part of speech assignment taking into account the entire 
sentence; page 204, column 1, line 22.) 

8. Consider claim 13, A method of morphological analysis comprising (Japanese 
morphological analyzer, abstract): 

applying a prescribed method of morphological analysis to a text and generating 
one or more hypotheses as candidate results of the morphological analysis, each 
hypothesis being a word string with part-of-speech tags, the part-of-speech tags 
including form information for parts of speech having forms (We propose a novel search 
strategy for getting the N best morphological analysis hypotheses for the input 
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sentence; page 201 , column 1 , line 35. We used tile simple tri-POS model as the 
tagging model for Japanese; page 201, column 1, line 30.); 

calculating probabilities that each said hypothesis will appear in a large corpus of 
text by using a weighted combination of a plurality of part-of-speech n-gram models, at 
least one of the part-of-speech n-gram models including information about forms of 
parts of speech (equations 3, 4 and 5 described in section 201 estimates the 
probabilities of the tagging with relation to the relative frequencies of the corresponding 
events.); and 

finding a solution among said hypotheses, based on said probabilities (Once 
word hypotheses for unknown words are generated, the proposed N-best algorithm will 
find tile most likely word segmentation and part of speech assignment taking into 
account the entire sentence; page 204, column 1, line 22.). 

9. Consider claim 20, Nagata teaches a machine-readable medium storing a 
program comprising instructions that can be executed by a computing device to carry 
out morphological analysis by the method of claim 1 3 (Nagata teaches a morphological 
analyzer; abstract. In order to operate this analyzer it is inherent that there is in fact ca 
storage medium containing the instructions in order to operate it. ). 
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Claim Rejections - 35 USC § 103 

10. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

1 1 . The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1 , 148 
USPQ 459 (1966), that are applied for establishing a background for deternnining 
obviousness under 35 U.S.C. 103(a) are summarized as follows: . 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating 
obviousness or nonobviousness. 



12. Claims 2, 3, 14, and 15 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Nagata. 



13. Consider claim 2, Nagata teaches the morphological analyzer of claim 1, but 
does not specifically teach wherein said at least one of the part-of-speech n-gram 
models including information about forms of parts of speech is a hierarchical part-of- 
speech n-gram model. 

However Nagata's language model does suggest using hierarchical part-of- 
speech n-gram model as the model takes into account the frequency that a given word 
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is tagged a given part of speech; page 202, column 1 , line 6. As one possible tag will 
have a higher value than another tag, then this model can be considered hierarchical. 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to recognize that the morphological analyzer as taught by Nagata could 
be use a hierarchical speech model in order to provide a more accurate language model 
to the analyzer. 

14. Consider claim 3, Nagata teaches the morphological analyzer of claim 2, wherein 
the hierarchical part-of-speech n-gram model calculates a product of a conditional 
probability P(wi I ti) of occurrence of a word wi given its part of speech ti, a conditional 
probability P(tiform I tipos) of occurrence of the part of speech tipos of said word wi in a 
form tiform shown by said word wi, and a conditional probability P(tiPos I ti-N+l...ti-l) of 
occurrence of the part of speech tipos of said word wi following a part-of-speech tag 
string ti-N+l...ti-i indicating parts of speech of N - 1 preceding words, where N is a 
positive integer (If we have some tagged text available, we can estimate the 
probabilities P(tdti_2,tiJ ) and P(wiltl) by computing the relative frequencies of the 
corresponding events on this data: 
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where f indicates the relative frequency, N(w, t) is the number of times a given word w 
appears with tag t, and N(ti_2,ti-1 ,tl) is the number of times that sequence ti-2 ti-2,ti 
appears in the text. It is inevitable to suffer from sparse-data problem in the part of 
speech tag trigram probability; page 202, column 1). 

15. Consider claim 14, Nagata teaches method of claim 13, but does not specifically 
teach wherein said at least one of the part-of-speech n-gram models including 
information about forms of parts of speech is a hierarchical part-of-speech n-gram 
model. 

However Nagata's language model does suggest using hierarchical part-of- 
speech n-gram model as the model takes into account the frequency that a given word 
is tagged a given part of speech; page 202, column 1 , line 6. As one possible tag will 
have a higher value than another tag, then this model can be considered hierarchical. 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to recognize that the morphological analyzer as taught by Nagata could 
be use a hierarchical speech model in order to provide a more accurate language model 
to the analyzer. 

16. Consider claim 15, Nagata teaches the method of claim 14, wherein the 
hierarchical part-of-speech n-gram model calculates a product of a conditional 
probability P(wi I ti) of occurrence of a word wi given its part of speech ti, a conditional 
probability P(tiform I tipos) of occurrence of the part of speech tipos of said word wi in a 



Application/Control Number: 1 0/81 2,000 Page 9 

Art Unit: 2626 

form tiform shown by said word wi, and a conditional probability P(tiPos I ti-N+l...ti-l) of 
occurrence of the part of speech tipos of said word wi following a part-of-speech tag 
string ti-N+l...ti-i indicating parts of speech of N - 1 preceding words, where N is a 
positive integer (If we have some tagged text available, we can estimate the 
probabilities P(tdti_2,tiJ ) and P(wiltl) by computing the relative frequencies of the 
corresponding events on this data: 



where f indicates the relative frequency, N(w, t) is the number of times a given word w 
appears with tag t, and N(ti_2,ti-1 ,tl) is the number of times that sequence ti-2 ti-2,ti 
appears in the text. It is inevitable to suffer from sparse-data problem in the part of 
speech tag trigram probability; page 202, column 1.). 

17. Claims 4-7, and 16 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Nagata in view of Pla et a! (Improving Part-of-Speech Tagging using Lexicalized 
HMMs). 
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18. Consider claim 4, Nagata teaches the morphological analyzer of claim 1 , but 
does not specifically teach wherein at least one of the part-of-speech n-gram models is 
a lexicalized part-of-speech n-gram model. 

In the same field of part of speech tagging, Pla teaches at least one of the part- 
of-speech n-gram models is a lexicalized part-of-speech n-gram model (We introduce a 
simple method to build Lexicalized Hidden Markov models for improving precision of 
part-of-speech tagging; abstract.). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to used lexicalized HMMs as taught by Pla with the analyzer of Nagata 
in order to improve tagging accuracy. 

19. Consider claim 5, Pla teaches the morphological analyzer of claim 4, wherein the 
lexicalized part-of-speech n-gram model calculates a product of a conditional probability 
P(wi I ti) of occurrence of a word wi given its part of speech ti and a conditional 
probability P (ti I Wi-N+1ti-N+l ... Wi-1 ti-1) of occurrence of the part of speech ti of said 
word wi following N - 1 words wi-N+1 wi-1 having respective parts of speech ti-N+1... ti- 
1 , where N is a positive integer (Lexical probabilities P(wi|ci) are calculated by dividing 
the frequency of the pair <wi,ci> by the frequency of the category ci. Contextual 
probabilities for trigrams are estimated by dividing the frequency of the sequence <ci, ci- 
1 , ci-2> by the frequency of sequence < ci-1 . ci-2>; page 7 line 23.). 
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20. Consider claim 6, Pla teaches the morphological analyzer of claim 4, wherein the 
lexicalized part-of-speech n-gram model calculates a conditional probability P(witi I ti- 
N+1 ...ti-1 ) of occurrence of a word wi having a part of speech ti following a string of N - 
1 parts of speech ti-N+1 ..-ti-1, where N is a positive integer integer (Lexical probabilities 
P(wi|ci) are calculated by dividing the frequency of the pair <wi,ci> by the frequency of 
the category ci. Contextual probabilities for trigrams are estimated by dividing the 
frequency of the sequence <ci, ci-1 , ci-2> by the frequency of sequence < ci-1 . ci-2>; 
page 7 line 23). 

21 . Consider claim 7, Pla teaches the morphological analyzer of claim 4, wherein the 
lexicalized part-of-speech n-gram model calculates a conditional probability P{witi I wi- 
N+1 ti-N+l. . . Wi-1 ti-1 ) of occurrence of a word wi having a part of speech ti following a 
string of N - 1 words wi-+1 ...wi-i having respective parts of speech ti-N+1 ...ti-1 , where N 
is a positive integer (Lexical probabilities P(wi | ci) are calculated by dividing the 
frequency of the pair <wi,ci> by the frequency of the category ci. Contextual 
probabilities for trigrams are estimated by dividing the frequency of the sequence <ci, ci- 
1 , ci-2> by the frequency of sequence < ci-1 . ci-2> page 7 line 23. For the model, each 
state will take into account the previous word and tag of the previous state.) 

22. Consider claim 16, Nagata teaches the method of daim 13, but does not 
specifically teach wherein at least one of the part-of-speech n-gram models is a 
lexicalized part-of-speech n-gram model. 
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In the same field of part of speech tagging, Pla teaches at least one of the part- 
of-speech n-gram models is a lexicalized part-of-speech n-gram model (We introduce a 
simple method to build Lexicalized Hidden Markov models for improving precision of 
part-of-speech tagging; abstract.). 

Therefore It would have been obvious to one of ordinary skill in the art at the time 
of the invention to used lexicalized HMMs as taught by Pla with the analyzer of Nagata 
in order to improve tagging accuracy. 

23. Claims 8 and 17 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Nagata in view of Abrego et al (US PAP 2003/0046078). 

24. Consider claim 8, Nagata teaches the morphological analyzer of claim 1 , but 
does not specifically teach wherein at least one of the part-of-speech n-gram models 
stored in the model storage facility is a class part-of-speech n-gram model. 

In the same field of n-gram language modeling, Abrego teaches at least one of 
the part-of-speech n-gram models stored in the model storage facility is a class part-of- 
speech n-gram model (It is important to notice that the resulting N-grams from the N- 
gram generator 18 are not word N-grams but instead class N-grams. The N-grams of 
valid classes 20 produces estimations of the probability that a class will follow its given 
history or sequence in an N-gram; i.e., how likely a sequence of classes are to be valid; 
paragraph 0038.). 
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Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to use class n-gram models as taught by Abrego for parsing as taught 
by Nagata in order to provide a more accurate part of speech analysis. 

25. Consider claim 1 7, Nagata teaches the method of claim 1 3, but does not 
specifically teach wherein at least one of the part-of-speech n-gram models stored in 
the model storage facility is a class part-of-speech n-gram model. 

In the same field of n-gram language modeling, Abrego teaches at least one of 
the part-of-speech n-gram models stored in the model storage facility is a class part-of- 
speech n-gram model (It is important to notice that the resulting N-grams from the N- 
gram generator 18 are not word N-grams but instead class N-grams. The N-grams of 
valid classes 20 produces estimations of the probability that a class will follow its given 
history or sequence In an N-gram; i.e., how likely a sequence of classes are to be valid; 
paragraph 0038.). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to use class n-gram models as taught by Abrego for parsing as taught 
by Nagata in order to provide a more accurate part of speech analysis. 

26. Claim 9-1 1 , and 18 rejected under 35 U.S.C. 103(a) as being unpatentable over 
Nagata and Abrego as applied to claim 8 above, and further in view of Pla et al. 
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27. Consider claim 9, Nagata and Abrego teaches the morphological analyzer of 
claim 8, but does not teach specifically wherein the class part-of-speech n-gram model 
calculates a product of a conditional probability P(wi I ti) of occurrence of a word wi 
given its part of speech ti and a conditional probability P (ti I Ci-N+1 ti-N+1 ... Ci-1 ti-1) of 
occurrence of said part of speech ti following a string of N - 1 words assigned to 
respective classes Ci-N+1 ...Ci-1 with respective parts of speech ti-N+1 ...ti-1 . where N is 
a positive integer. 

In the same field of tagging, Pla teaches lexical part-of-speech n-gram model 
calculates a product of a conditional probability P(wi I ti) of occurrence of a word wi 
given its part of speech ti and a conditional probability P (ti I Ci-N+1 ti-N+1 ... Ci-1 ti-1) of 
occurrence of said part of speech ti following a string of N - 1 words assigned to 
respective classes Ci-N+1. ..Ci-1 with respective parts of speech ti-N+1... ti-1, where N is 
a positive integer (Lexical probabilities P(wi | ci) are calculated by dividing the frequency 
of the pair <wi,ci> by the frequency of the category ci. Contextual probabilities for 
trigrams are estimated by dividing the frequency of the sequence <ci, ci-1 , ci-2> by the 
frequency of sequence < ci-1 . ci-2>; page 7 line 23). 

Therefore it would have been obvious to apply the same rules of lexical tagging 
to the class tagger of Nagata and Abrego in order to provide a more accurate tagging, 
taking fully into consideration the class of which the words belong. 

28. Consider claim 10, Nagata and Abrego teaches the morphological analyzer of 
claim 8, but does not specifically teach wherein the class part-of-speech n-gram model 
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calculates a product of a conditional probability P(witi I Ci-N+1 ti-N+1...Ci-1ti-1) of 
occurrence of a word wi having a part of speech ti following a string of N - 1 words in 
respective classes Ci-N+I...Ci-l With respective parts of speech ti-N+1...ti-1, where N is 
a positive integer. 

In the same field of tagging, Pla teaches wherein the lexical part-of-speech n- . 
gram model calculates a product of a conditional probability P(witi I Ci-N+1 ti-N+1...Ci- 
1ti-1 ) of occurrence of a word wi having a part of speech ti following a string of N - 1 
words in respective words Ci-N+1. ..Ci-I With respective parts of speech ti-N+1...ti-1, 
where N is a positive integer (Lexical probabilities P(wi | ci) are calculated by dividing 
the frequency of the pair <wi,ci> by the frequency of the category ci. Contextual 
probabilities for trigrams are estimated by dividing the frequency of the sequence <ci, ci- 
1 , ci-2> by the frequency of sequence < ci-1 . ci-2>; page 7 line 23 For the model, each 
state will take into account the previous word and tag of the previous state.) 

Therefore it would have been obvious to apply the same rules of lexical tagging 
to the class tagger of Nagata and Abrego in order to provide a more accurate tagging, 
taking fully into consideration the class of which the words belong. 

29. Consider claim 1 1 , Nagata and Abrego teaches the morphological analyzer of 
claim 8, but does not specifically teach wherein the class part-of-speech n-gram model 
is trained from both a part-of-speech tagged corpus and a part-of-speech untagged 
corpus. 
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In the same field of n-gram models, Pla teaches the class part-of-speech n-gram 
model is trained from both a part-of-speech tagged corpus and a part-of-speech 
untagged corpus (The learning process of the parameters can be carried out from 
labeled corpora or an unlabelled corpus; page 7, line 15.). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to combine learning method of Pla with the analyzer of Abrego and 
Nagata In order to allow for a more robust final model after training. 

30. Consider claim 18, Nagata and Abrego teaches method of claim 17, but does not 
specifically teach wherein the class part-of-speech n-gram model is trained from both a 
part-of-speech tagged corpus and a part-of-speech untagged corpus. 

In the same field of n-gram models, Pla teaches the class part-of-speech n-gram 
model is trained from both a part-of-speech tagged corpus and a part-of-speech 
untagged corpus (The learning process of the parameters can be carried out from 
labeled corpora or an unlabelled corpus; page 7, line 15.). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to combine learning method of Pla with the analyzer of Abrego and 
Nagata in order to allow for a more robust final model after training. 

31. Claim 12 is rejected under 35 U.S.C. 103(a) as being unpatentable over Nagata 
in view of Siu (Variable N-Grams and Extensions for Conversational Speech Modeling). 
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32. Consider claim 12, Nagata teaches the morphological analyzer of claim 1, but 
does not specifically teach further comprising a weight calculation unit using a leave- 
one-out method to calculate weights of the part-of-speech n-gram models. 

In the same field of part of speech tagging, Siu teaches a weight calculation unit 
using a leave-one-out method to calculate weights of the part-of-speech n-gram models 
(To evaluate the effect of node pruning and merging on unseen data, we use leave-one- 
out (LOO) likelihood to estimate the distributions used in the distance measures; column 
69, column 2, section B.). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to use the leave one out method as taught by Siu with the analyzer of 
Nagata in order to provide a more robust method of training. 

33. Consider claim 19, Nagata teaches the method of claim 13, but does not 
specifically teach further comprising a weight calculation unit using a leave-one-out 
method to calculate weights of the part-of-speech n-gram models. 

In the same field of part of speech tagging, Siu teaches a weight calculation unit 
using a leave-one-out method to calculate weights of the part-of-speech n-gram models 
(To evaluate the effect of node pruning and merging on unseen data, we use leave-one- 
out (LOO) likelihood to estimate the distributions used in the distance measures; column 
69, column 2, section B.). 
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Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to use the leave one out method as taught by Siu with the analyzer of 
Nagata in order to provide a more robust method of training. 

Conclusion 

34. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure is listed on the Notice of References Cited. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Douglas C. Godbold whose telephone number is (571) 
270-1451 . The examiner can normally be reached on Monday-Thursday 7:00am- 
4:30pm Friday 7:00am-3:30pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Patrick Edouard can be reached on (571) 272-7603. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you mvoM like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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